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REMARKS 

This paper is responsive to the Office Action of April 19, 2007. 
Reconsideration and allowance of claims 3, 4, 6, 14-18, and 20-30 are requested. 

The Office Action 

Claims 3, 4, 5, 6, 8, and 14-16 stand rejected under 35 U.S.C. § 102 as 
being anticipated by Harris (US 2005/0052423). 

The Claims Are Not Anticipated by Harris 

The present application was filed October 10, 2000. The Harris 
application was filed May 4, 2004 as a continuation of an application filed on 
March 12, 2001, i.e., Harris has a filing date about five months after the filing date of 
the present application. Harris claims the benefit of provisional application 
60/189,487 which was filed prior to the present application. However, the Harris 
provisional application is quite different from the Harris published application and 
only discloses a fraction of the subject matter. 

It is axiomatic that Harris is only a valid reference against the present 
application to the extent the subject matter is disclosed in an enabling manner in the 
provisional application. Subject matter disclosed for the first time in the Harris 
non-provisional applications is not prior art against the present application by virtue 
of its later filing date. It is suggested that in future Office Action, if the Examiner 
continues to apply Harris, that the Examiner cite to page and line references in the 
provisional application. After all, anything which is not disclosed in the provisional 
application is not a reference against the present application. 

Claim 3 is not anticipated by Harris. First, claim 3 calls for the data to be 
in a mark-up language format. The Harris provisional application makes no 
suggestion of using mark-up language format. Accordingly, any reference to a mark- 
up language format in the Harris published application does not antedate the filing 
date of the present application and is not prior art. 

Claim 3 further calls for the mark-up language code set that is received 
over the internet to include a control code and a GUI panel. This finds antecedent 
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basis in the present application in the first full paragraph of page 7. The Harris 
provisional application makes no suggestion of downloading such a GUI panel. 

Claim 3 further calls for enabling the control device to display the GUI 
panel corresponding to the specified apparatus on a touch screen. This finds 
antecedent basis in original claim 8 and the last full paragraph on page 2 of the 
application. The Harris provisional application makes no suggestion of displaying a 
GUI panel on a touch screen. Accordingly, it is submitted that claim 3 and 
claims 4 and 6 dependent therefrom are not anticipated by Harris. 

Claim 14 calls for a remote control device which receives a code set that 
includes a control code and a GUI element. The Harris provisional application makes 
no suggestion of this. 

Claim 14 further calls for a display panel and for the remote control 
device to be configured to convert the GUI element into a graphical representation on 
the display panel of the apparatus's remote control. The Harris provisional 
application makes no suggestion of either a GUI element or using it to generate such a 
graphical representation. 

Accordingly, it is submitted that claim 14 and claims 20, 21, and 27 are 
not anticipated by Harris. Claim 8, which subject matter has now been incorporated 
into claim 14, has been cancelled. 

Claim 15 calls for control codes which are formatted in a mark-up 
language. The Harris provisional application makes no suggestion of codes in a 
mark-up language. Accordingly, it is submitted that claim 15 and claims 22 and 28 
dependent therefrom are not anticipated by Harris. 

Claim 16 calls for a control code which is supplied in an XML format. 
The Hams provisional application does not disclose an XML format. Accordingly, 
claim 16 is not anticipated by Harris. 

Claim 17 calls for the control included in data in a mark-up language 
format. The Harris provisional application does not disclose data in a mark-up 
language format. Accordingly, claim 17 and claims 23, 24, and 29 dependent 
therefrom are not anticipated by Harris. 

Claim 29 calls for the mark-up language format data to further include a 
GUI element. The Harris provisional application fails to disclose mark-up language 
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including a GUI element. Accordingly, for these further reasons it is submitted that 
claim 29 is not anticipated by Harris. 



Claim 18 calls for control codes in a mark-up language format. The 



Harris provisional application fails to disclose such mark-up language format codes. 
Claim 18 further calls for such mark-up language format codes to include a first set 
which is part of an EPG or ECG and a second set representing commands for IR or 
RF transmission. Claim 30 further adds a third set including a GUI element. 



Because these features are not disclosed in the Harris provisional 



application, it is submitted that claim 18 and claims 25, 26, and 30 dependent 
therefrom are not anticipated by Harris. 

The Harris Provisional Application 

Although a complete copy of the Harris provisional application 
60/189,487 is available to the Examiner on Public PAIR, the applicants enclose 
another copy of this non-provisional application for the Examiner's convenience. 



For the reasons set forth above, it is submitted that claims 3, 4, 6, 14-18, 



and 20-30 are not anticipated by Harris. An early allowance of all claims is 
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A System, Method and Apparatus for an Internet Enabled User 
Interaction Device 



Background of the Invention 

Many households currently contain devices that can be operated using a remote control Examples of such 
devices include: 

• Home entertainment systems (televisions, VCRs 9 Web TV, etc.) 

• Home appliances (air conditioning systems, garage door openers, etc.) 

• Home automation equipment (automatic lights, automatic blinds, etc.) 

• Home computers (control centers, music repositories, etc.) 

Due to the large number of devices, households often have many remote controls. When a consumer 
wishes to operate a given device, it is often difficult for the user to: 

1 . Find the correct remote control 

2. Find the correct button 

3. Determine the correct sequencing (if multiple devices are operated together) 

In response to this confusion, many manufacturers currently make ^universal' remote controls that are 
designed to provide the functionality of a number (normally 3-7) of dedicated remote controls. Universal 
remote controls are a partial solution to the problem, but have a number of drawbacks: 

• They can be difficult to program and/or customize. For example, to configure the universal remote 
control the user often has to find the particular code that corresponds to their device (by searching a 
document, or by searching the Internet), and then enter that code using buttons on the remote control 

• It can be difficult to remember which button performs which function. For example, when watching 
television, the Tiay ? button may bring activate the 'Picture in Picture' functionality. 

• They often have a number of buttons to provide the 'superset* of button functionality, leading to a 
large, ugly, confusing device, 

In response to the above drawbacks, new types of universal remote controls have begun appearing on the 
market, These remote controls typically feature large touch sensitive LCD displays, and contain powerful 
processors to generate arbitrary user interfaces, and interpret the user's input. In general, these remote 
controls are very expensive (more than US $200), and can be difficult to operate and/or customize. 

With increasing access to powerful computers and Internet connections in average households, the average 
consumer now has access to a large amount of information via the Internet. A large amount of that 
information is related to the interactions that consumers have with their home entertainment systems (for 
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example, television schedules, information about products advertised on television), yet the computer is 
almost always in a different room than the home entertainment system. Devices such as WebTV boxes 
promise to merge the power of the Internet with home entertainment, but so far these devices have been 
greeted with limited consumer adoption. 

What is currently needed is a small, very inexpensive user interaction device that is able to bring the power 
of the Internet into the living room, This device is the Internet Enabled User Interaction Device. 

Summary of the Invention 

Accordingly, the main object of the present invention is to provide a novel user interaction device that is 
inexpensive and can provide the user with additional information relevant to operation of devices 
controlled by the user interaction device. 

Another object of the present invention is to provide a novel remote control that can control a large range of 
devices, yet is simple for an average consumer to both configure and operate. 
Yet another object of the present invention is to provide the user with customized and interactive 
information relevant to the usage of the user interaction device (for example, a television guide). 
Yet another object of the present invention is to provide a means for the user to register feedback on 
content without requiring modifications to either the content or the devices used to experience the content. 
Another object of the present invention is to provide a novel remote control that is small in size, yet can 
control all aspects of device fimctionaiity- 

Another object of the present invention is to allow the user to use the functionality of a home computing 
device to configure the novel remote control in an intuitive manner. 

Another object of the present invention is to provide a novel user interaction device that requires only 
limited processing abilities, and therefore can be manufactured more cheaply that competing user 
interaction devices. 

Another object of the present invention is to provide a novel user interaction device that provides relevant 
information to the user without requiring a continuous connection to that information (for example, the 
Internet). 

Another object of the present invention is to provide a novel user interaction device that can synchronize 
with information services on an irregular basis, and provide a quality of information to the user that 
degrades gracefully over time (for example, if the user synchronizes on Monday, then the user interaction 
device may contain detailed programming information for all shows during the coming week, but for the 
next week may only contain information about recurring shows). 

These and other objects and advantages of the present invention are accomplished by providing a novel 
Internet Enabled User Interaction Device that is compatible with existing home entertainment products. 
The Internet Enabled User Interaction Device is configured using a separate computing device (for 
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example, a computer, or WebTV device) that provides a simple interface for the average user. The 
Internet Enabled User Interaction Device is configured in such a way as to know the user's preferences (for 
example, the user normally watches "Fox 29" and likes the show 'The X-Files"). The Internet Enabled 
User Interaction Device can be updated with information about content available to the user, and can 
provide an interactive, customized guide to that content using a simple interface. 

Brief Description of the Drawings 

FIG. 1 shows a schematic block diagram of a preferred embodiment of the apparatus for carrying out the 
method according to the present invention. 

FIQ 2 shows a preferred embodiment of an apparatus for carrying out the method according to the present 
invention. 

Detailed Description 

FIG. 1 shows a schematic block diagram of a configuration of modules used to carry out the method of the 
present invention. A processor module 1 is connected to the interface communication module 2, the device 
communication module 3, the environment input module 4 S the user input module 5, the user output module 
6, the code storage module 7 and the data storage module 8. The power module 9 is connected (not shown) 
to every module that requires power. The device communication module 3 is able to communicate with 
some or all of the devices 14. l~14,n (where n is an integer to indicate a range from zero to many). The 
Internet Enabled User Interaction Device (IEUID) 50 contains some or all of the modules 1 to 9. The 
interface communication module 2 is able to communicate with the Interface System 10, which is 
connected to the Network 1 1. Also connected to the Network 1 1 is the Configuration Information System 
20, the Data Collection System 21 and the Content Information System 22. 

It is noted that according to the present invention the grouping of functionality into modules is arbitrary, 
and is done for illustrative purposes only. 

It is noted that according to the present invention each module can be composed of a plurality of modules. 
For example, the device communication module 3 can be composed of an infrared receiver module, an 
infrared transmitter module and a radio-frequency transmitter module, 

It is noted that one physical component can perform the functions of one or more modules. For example, 
one integrated circuit (IC) can be constructed to incorporate the processor module 1 and the interface 
communication module 2. 

It is noted that the interface communication module 2 may use any appropriate technology that can 
communicate with the Interface System 10, including but not limited to; 

• Wireless (for example, IrDA, BlueTooth) 

* Wired (for example, RS232 serial, Universal Serial Bus (USB)) 
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• Networking {for example, the 3EUDD communicates via TrDA to a Network Interface Device that 
communicates via a home network to the interface system 10) 

It is noted that the interface communication module 2 may allow the 1EUID to function as a Network 
Interface Device, in which case the Interface System 10 is optional. 

It is noted that the device communication module 3 may use any appropriate technology that can 
communicate with the devices 14. l-14.x ? including but not limited to; 

• Infrared 

• Radio-frequency 

• Ultrasonic 

It is noted that the device communication module 3 may include unidirectional (for example, an infrared 
transmitter) and bidirectional (for example, an infrared transmitter and receiver) capabilities. 
It is noted that the device communication module 3 may also perform the functions of the interface 
communication module 2. 

It is noted that the environment input module 4 may use any appropriate technology that senses the 
environment surrounding the IEUID, including but not limited to: 

• Light detectors 

• Motion and orientation detectors 

• Temperature detectors 

• Audio detectors 

It is noted that the user input module 5 may use any appropriate technology that can accept input from the 
user, including but not limited to: 

• Binary input devices (for example, pushbuttons) 

• Analog input devices (for example, dials) 

• Motion and orientation detectors 

• Touch sensitive screens 

• Voice recognition 

It is noted that the user output module 6 may use any appropriate technology that can communicate some or 
all of the status of the IEUID to the user, including but not limited to: 

• Displays (for example, Liquid Crystal Displays (LCDs) or Cathode Ray Tubes (CRTs)) 

• Lights (for example, Light Emitting Diodes (LEDs)) 

• Sound emitting devices (for example, buzzers or speakers) 

• Tactile devices (for example, a vibrator) 
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It is noted that the status of the IEUID may be communicated to the user by using the device 
communication module 3 to direct any or all of the devices 14. l-14.x appropriately. For example, the 
IEUID may direct the television to display the status of the IEUID on a text overlay. 

It is noted that the code storage module 7 and data storage module 8 may use any appropriate technology 
(removable or non-removable) including but not limited to: 

• Random Access Memory (for example, Flash or SRAM) 

• Magnetic storage (for example, an IBM MicroDrive) 

• Optical storage (for example, MiniDisc) 

It is noted that the power module 9 may use any appropriate technology, including but not limited to: 

• Batteries 

• Solar power 

• Input from an external power source 

It is noted that the power module 9 can deliver power to each module according to the requirements of the 
module. For example, the power module 9 can deliver 3 volts to the processor 1 , and 5 volts to the user 
output module 6. 

It is noted that the Interface System 10 may be any device that can communicate with the IEUID, including 
but not limited to: 

• Computer 

• WebTV device 

• Personal Digital Assistant (PDA) 

• Cellular telephone 

It is noted that the connection between the Interface System 10 and the Network 1 1 is optional, and may 
use any appropriate technology that allows communication between Network Interface Devices, including 
but not limited to: 

• Wired communication (for example, cable modem broadband access) 

• Wireless communication (for example, BlueTooth) 

• Dialup access through an Internet Service Provider (ISP) 

• Hybrid services (for example, combined dialup and out-of-band television signaling) 

It is noted that a Network is a grouping of Network Interface Devices that can communicate with each 
other. The grouping of devices may change over time, and may consist of a homogenous or heterogeneous 
set of Network Interface Devices. A Network may physically connect Network Interface Devices together 
(for example, with electrons traveling over copper wire) or may use other means to enable 
intercommunication (for example, radio waves). 
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It is noted that a Network Interface Device is a device that allows an entity (for example, a person or 
system) to interact with other entities that are connected to Network Interface Devices on the same 
Network. 

It is noted that the Network 1 1 may be any technology that can interconnect multiple Network Interface 
Devices, including but not limited to: 

♦ The Internet 

♦ Local Area Network 

• Wide Area Network 

• Computer bus (for example, ISA, PCI, USB bus) 

It is noted that the plurality of devices 14. l-14.n (also known as the Device Network) may or may not be 
interconnected. For example, the Device Network may consist of an infrared controlled radio receiver, and 
an infrared controlled ceiling fen. 

It is noted that the devices 14. 1-14 jn that make up the Device Network are not required to be controlled 
using the same communication means. For example, the Device Network may consist of an infrared 
controlled television that is connected to a radio-frequency controlled satellite receiver. 
It is noted that some or all of the devices 14_l-14.n may be interconnected using control mechanisms 
different to those used by the IEUID. For example, a Sony television and a Sony videocassette recorder 
may be connected using the Sony S-Link control bus. 

It is noted that the Configuration Information System 20 is any Network Interface Device that stores 
programming information and techniques used when programming the IEUID. 

It is noted that the Data Collection System 21 is any Network Interface Device that stores information 
uploaded to it by the IEUID (via the Interface System 10 and Network 1 1), For example, the IE LID may 
be configured to upload cable television viewing statistics to the Data Collection System 21 every time the 
IEUID is connected to the Interface System 10. 

It is noted that the Content Information System 22 is any Network Interface Device that can provide 
information about content to the IEUID (via the Interface System 10 and Network 1 1). For example, the 
IEUID may be configured to download television programming schedules from the Content Information 
System 22 every time the IEUID is connected to the Interface System 10. 

FIG. 2 shows one preferred embodiment of an apparatus for carrying out the method according to the 
present invention. Interface communication is provided through Universal Send Bus (USB) connector 
202. 1 and USB chip 202.2. Device communication is provided through infrared transmitter/receiver 203 . 1 . 
Environment input is provided through light sensor 204*1, User input is provided through rocker switch 
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205.1 and pushbuttons 205.2-205. 15, User output is provided through a backlit LCD 206.1 and internal 
speaker 206.2. Code storage is integrated with the processor 201 (which also includes a real time clock). 
Data storage is provided through flash memory chip 208. L Power is provided through AA battery 209. 1 
Connection between the various modules is provided through a combination of electrical wiring and printed 
circuit boards. 



It is noted that the LCD can be used in combination with the switch 205.1 and pushbutton 205.2 to provide 
an interactive user interface that allows the user to activate arbitrary functionality of the IEUID, and to 
display arbitrary information that is accessible by the IEUID. 

It is noted that the configuration information stored by the IEUID in the data module 8 is not restricted in 
any way, and includes (but is not limited to) categories of information such as: 
Device. This information records which devices are to be controlled and their associated control 
mechanisms. Examples of this type of information include: 

• The fact that the user wishes to control a device network that just consists of a Sony CDP-CX255 
compact disc player, and the infrared control information required to activate its fimctionality. 

• The feet that the user wishes to control a device network that consists of a Sony CDP-CX255 compact 
disc player which is connected to a Sony STR-DE905 receiver, and the infrared control and sequencing 
information required to activate their combined functionality. 

Usability. This information records the preferences of the user regarding the access to IEUID functionality. 
Examples of this type of information include: 

• The user wishes the first menu option to be 'Play a videotape* 

• The user wishes button * A' to function as 'Watch television channel 50' at all times 
Preference . This information records the preferences of the user regarding the content that they are 
exposed to. Examples of this type of information include: 

• The user enjoys watching cable television channels 37 and 50. 

• The user never watches channel 24, 

• The user enjoys the television program 'The X-Files'. 

• The user never watches television 'talk show' programs. 

Usercontent. This information records information about the user controlled content that the user has 
access to via the IEUID. Examples of this type of information include: 

• The user has a 200 disc CD changer that contains the CD 4 U2 - Achtung Baby' as disc 1 and 4 Andrea 
Bocelli - Romamsr as disc 37, along with relevant information about the songs on each disc. 

• The user has a MP3 player that has the song 'Garbage - Only Happy When it Rains* as song 1 and 
'David Bowie - Blue Jean 5 as song 361, along with relevant information about each song. 

Non-user content. This information records information about the externally controlled content that the 
user has access to via the IEUID. Examples of this type of information include: 
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• Cable television channel 50 is playing the movie Terminator T on January 1, 2000 from 8pm to 
11pm. 

• Radio station FM 102.1 has a jazz show that plays every Sunday night from 5pm to 6pm, 

• Satellite television channel 37 is playing the miniseries 'Roots' at 8pm for the next four nights. 

• Sports scores and statistics. 

Interaction . This information records information about the interactions between the user and the IEUID. 
Examples of this type of information include: 

• The user configured the Device Network to display cable television for 23 hours on January 1 , 2000. 

• The user played four videotapes this week. 

• The user selected answer £ A' to the 'Who Wants to be a Millionaire' question at 7:23pm on January 1, 

2000. 

It is noted that a given piece of content (for example, a particular showing of a cable television show) may 
be described by multiple pieces of content information, For example, the IEUID may be programmed with 
the fact that cable television channel 50 plays a movie every Saturday night from 8pm to 1 1pm, and that the 
movie on the January 1, 2000 at that time is 'Terminator 2\ If the user views the schedule on January 1, 
2000 the IEUID may display information such as '8pm: Movie - Terminator 2\ whereas if the user views 
the schedule on January 8, 2000 the IEUID may display information such as '8pm: Movie', 

It is noted that any date related information stored by the IEUID may be stored either in an absolute format 
(for example, the movie is playing on January 1, 2000 at 8pm) or in relative format (for example, the movie 
is playing tomorrow night at 8pm, or the movie will being playing in 4 hours time). 
It is noted that if a real time clock is required by the IEUID then that functionality may be provided by 
either the processor module 1, or by connecting the processor module 1 to a real time clock circuit. 

It is noted that the functionality of the IEUID may be combined with that of other devices. For example, 
IEUID functionality may be incorporated into a cellular telephone, PDA or laptop via the addition of an 
infrared transmitter and custom software development- 
It is noted that any information stored by the IEUTD (for example, preference and interaction information) 
may be communicated to the Interface System 10. This information can be processed by systems such as 
the Interface System 10, Configuration Information System 20 or the Data Collection System 21 for many 
purposes, including but not limited to: 

• Allowing a cable television station to determine that what percentage of their viewers will switch to a 
competing channel if the first advertisement in an advertisement break is not humorous. 

• Allowing a device maker to determine what percentage of device users regularly use a particular 
feature of that device. 
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* Allowing a parent to determine how many hours of television their children are watching (and what 
shows). 

• Enabling market research firms to determine what percentage of households that watch The X-Files' 
also watch 'The Outer Limits'. 

• Enabling market research firms to determine what sports types are preferred, or what teams are 
followed by viewers. 

♦ Enabling a DVD manufacturer to email all users that frequently watch videos, but do not yet own a 
DVD player, with information about their latest promotion. 

It is noted that the IEUID may be used to obtain information from users regarding content that they have 
been exposed to. For example, if a user registers the fact that they enjoy watching the television show 
' Who Wants to be a Millionaire', then the IEUID may be configured to allow the user to answer questions 
using the IEUID during the show. The IEUID communicates with the Interfece System 10 to upload the 
user's answers to a Data Collection System 21, whereupon the user may be chosen to win a prize. 

The IEUID is a very flexible device that may be tailored in many ways. This process is known as 
configuration. During the configuration process, the data and code stored on the IEUID is altered to allow 
the device to satisfy the requirements and preferences of the user. 

Configuration of the IEUID is controlled by the Programming System, which is a conceptual computing 
system that may be distributed between any or all of the IEUID, the Interface System 10, the Programming 
Information System 20, the Data Collection System 21 and the Content Information System 22. In the 
following example, the Programming System is composed of the Configuration Information System 20 
(which is an Internet website) and the Inter&ee System 10 (which is a personal computer that is running 
both a web browser application and an IEUID communication application). The IEUID communication 
application relays requests from the website to the IEUID, and relays responses from the IEUID back to the 
website. If the website is attempting to determine the infrared control codes of a device 14.x owned by the 
user (where 14.x is within the range 14. 1-I4.n), then the website can request the IEUID to begin sampling 
from its infrared receiver. The website can use the web browser application to instruct the user to press a 
button on the remote control for device 14.x, and then receive the sampled information from the IEUID. 
Based on the sampled information, the website can then automatically configure the IEUID to operate 
device 14.x. 

The IEUID is a very flexible device that may be used to provide many types of information to the user. 
The process of updating the IEUID with the most recent information is known as synchronizing. During 
the synchronization process (which is also controlled by the Programming System), the data and code 
stored on the IEUID is altered to allow the device to provide the necessary information to the user. 
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It is noted that the configuration and synchronization processes may either be separate process, or may be 
combined into one operation- 
It is noted that the Programming System may use any additional information from any Network Interface 
Devices that are connected to the Network 1 1, or from any of the devices in the Device Network. For 
example, the Programming System may communicate with a weather recording system via the Internet and 
upload that information to the IEUID. 

It is noted that the IEUID may be used by the user in a variety of ways, including but not limited to: 

• Universal remote control. 

• Game system (for example, the IEUID may be configured with a series of questions for a dictionary' 
or 'You don't know Jack 9 game, and provide those questions to the user either visually or audibly. The 
IEUID may also enhance game play by controlling external devices such as VCRs and CD players to 
provide additional input to the players). 

• Recipe system (for example, the IEUID may be configured with recipe information and be used in the 
kitchen to display recipes for the user, and receive input about items that need to be purchased. In this 
example, the IEUID may not require a device communication module 3, but may instead be equipped 
with a printer for printing out shopping lists), 

• Home automation controller (for example, the IEUID may be used to control a XI 0 network via an 
internal RF transmitter). 

• Bidding and betting system. 

• Personal digital assistant (PDA). 

Illustrative Operation of the IEUID 

Device identification 

Example: The user wishes to specify which devices are to be controlled by the IEUID. The Programming 
System obtains information from the user such as the make and model of each device in his home stereo 
system, and uploads the appropriate configuration to the IEUID via the Interface System 10. 

It is noted that the Programming System can use any means to determine the devices in the user's system 
including, but not limited to: 

• Asking the user questions (for example, make, model number, device type, device physical 
appearance, remote control physical appearance) 

• Information obtained from the IEUID (for example, infrared codes emitted by the device remote 
controls) 
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• Showing pictures (for example, pictures of possible devices or device remote controls) 

It is noted that the Programming System may use an iterative technique to allow the user to narrow down 

the list of possible choices until the user's device is selected. 

It is noted that the IEUID can be used to digitize the emissions from the device or device remote control 
and can communicate this information to the Programming System (for example, the IEUID may contain 
an infrared detector and analog to digital conversion circuitry that allows it to digitize infrared codes 
emitted by a device remote control). 

It is noted that at each stage in the device identification process the user can be prompted with possible 
pieces of information that can help restrict the search. For example, the current selection may be between 
two devices which have the same infrared control code for function c Play\ but have differing infrared 
control codes for function 'Stop\ In such a case, the user would be prompted to press the device remote 
control 'Stop' button for the IEUID to digitize (rather than the Tlay* button). 

It is noted that with each new piece of information, the number of possible devices is restricted and the 
operation of the Programming System may be altered accordingly. For example, if there are 20 devices, 
then the user may be shown a list of matching descriptions. If there are 5 matching devices, then the user 
may be shown pictures of the matching devices. 

Configuration 

Example: The user wishes to configure the IEUID to operate the user's home entertainment system. Based 
on the devices previously identified by the user, the Programming System provides a default configuration 
for the IEUID that will allow the user to operate all components of the home entertainment system. 
Because the home entertainment system includes a television, the Programming System prompts the user to 
select which cable provider is being used. Because the home entertainment system includes a CD jukebox, 
then user ts asked if he wishes to enter CD title information. With each CD selected by the user, the 
Programming System uses the Internet to connect to a CD database server and obtain artist, title, and track 
information. The Programming System then provides the user with a list of common macros which the 
user may be interested in adding to his configuration. The Programming System then uses the Internet to 
connect to TVGuide.com and download the current television schedule. The Programming System then 
transfers all necessary code and data to the IEUID. Finally , the Programming System asks the user if he 
wishes to print out a booklet that describes the operation of the configured IEUID. 
Example: The user wishes to add the television channel "Global" to his list of preferred content. The user 
uses the Programming System to add the channel, and then the Programming System transfers the changed 
configuration to the IEUID. Now, when the user browses available television channels on the IEUID, the 
television channel "Global" is at the top of the list. 

Example: The user wishes to customize the operation of the IEUID. The user uses the Programming 
System to change the default television show reminder to be * Sound an alarm 1 minute before the show 
starts', and changes the navigation up key to select the function 'View cable television channel 50'. 
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It is noted that the Programming System may store the configuration information to allow the user to 
incrementally modify the configuration. For example, the user may add a new CD title to the CD jukebox. 
It is noted that the Programming System may be used to modify any aspect of the configuration (data 
and/or code) of the IEUED. For example, this can even include complete functionality changes where one 
user develops software designed to control a home automation system, and then allows other users to use 
the Programming System to configure their IEUIDs with the new software and associated data. 
It is noted that the Programming System may use any available information to assist the user in 
customizing the operation of the IEUED, including but not limited to: 

• Device Network Information 

• User Preferences 

• Sample configurations provided by other users. 

It is noted that the Programming System is designed to make the IEUID simple to configure for a novice 
user. For example, if the user has specified to the Programmer 10 that they wish to control a television and 
videocassette recorder, then the Programmer 10 may allow the user to select from common operations that 
are performed with systems containing a television and videocassette recorder Examples of such 
operations include 'Play a videotape' and 'Record a television show'. If the user chooses to accept such an 
operation, then the Programming System configures the IEUID in such a manner as to ensure that when the 
user selects the operation Tlay a videotape 5 then the IEUID sends control signals as required to direct the 
Device Network to perform the desired operation (if possible). 

It is noted that the IEUID may be purchased pre-configured to the user's preferences, in which case 
configuration via the Programming System is optional. 

It is noted that the IEUID may be configured using the user input module 5, user output module 6 and 
device communication module 3, in which case the IEUID is the sole component of the Programming 
System, and the connection to the Interface System 10 is optional. For example, the IEUID can be 
instructed to 'learn* a remote control code from an infrared remote control, as is well known in the art. 
It is noted that the IEUID may be configured using just the Interface System 10, in which case the Interface 
System 10 is sole component of the Programming System. 

It is noted that the Programming System may display arbitrary information to the user. For example, while 
the IEUID is communicating with the Programming System the user may be shown advertising based on 
the types of television shows that the user enjoys watching. 

It is noted that the user may use the IEUID to alter any aspect of the configuration of the IEUID. For 
example^ the user may instruct the IEUID to remember a sequence of commands and assign them to a new 
menu option. 

It is noted that the user may interact with the Programming System when the IEUID is not connected to the 
Programming System, in which case any changes will be transferred to the IEUID at a later date. 
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Synchronizing 

Example : The user wishes to update the television show scheduling information stored on the IEUID The 
user connects the IEUID to his personal computer, and visits the wwwEasyZapper. corn website. The 
Programming System then determines the relevant television show scheduling information, and uploads the 
information to the IEUID for later access by the user. 

Examp le: The user wishes to view the television show using his IEUID, but has not synchronized in the 
last two months. The IEUID contains information that describes television show time slots (for example, 
"The Simpsons" plays every Sunday at 6pm on "Global"), and is therefore able to provide the user with at 
least a partial television show listing. 

It is noted that if the IEUID has a continuous connection to the Interface System 10, then synchronization 
may take place automatically. For example, the IEUID may synchronize every night at 12pm, or every 
hour, or every minute. 

It is noted that the Programming System is not required to have a user interface if it is being used solely For 
synchronising the IEUID. 

It is noted that the synchronization information may include information taken from the Device Network. 
For example, the IEUID may query a FM radio receiver for the names of available radio stations. 

User Profiles 

Example: Mary and Joe share the same IEUID and wish to customize its operation according to their 
individual preferences. Each time one of them starts using the IEUID they indicate to the IEUID who they 
are, and the IEUID adjusts its operation according to their preferences. 

It is noted that the IEUID may use any means to determine which user is currently using the IEUID, 
including but not limited to: 

• User input (for example, the user selecting a menu item such as I am Joe*) 

• Processing heuristics (for example, Joe normally uses the IEUID from Sam to 5pm, while Bill 
normally uses the IEUID from 5pm to 12pm). 

It is noted that the user may have to validate their identity, for example by entering a password or key 
sequence. 

It is noted that any aspect of the IEUID can be affected by the identity of the current user, including but not 
limited to: 

• Restrictions (for example, 'Only user Joe can access the Cooking Channel*) 

• Preferences (for example, Joe and Mary have different "Tavorites" lists, Joe likes the IEUID to beep as 
a reminder when a television show is about to start) 
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Content preferences 

Example: As the user is watching television he is able to indicate to IEUID that "I like this show" or "I like 
this actor", and the IEUID records the user's preference. 

It is noted that the IEUID can record content preference information about any type of content including, 
but not limited to: 

• Content genre (for example, science fiction, rap) 

• Content author/publisher (for example, Stephen King, Paramount Studios) 

• Content artist/performer (for example, Mariah Carey, Clint Eastwood) 

• Content title (for example, "Seinfeld", "Howard Stern Morning Show") 

It is noted that the user is able to use the IEUID or Programming System to edit the content preference 
information in any way. 

Reminders 

Example: The user configures the IEUID to always remind him when "Seinfeld" is about to start. As the 
user is preparing dinner, the IEUID beeps to inform him that "Seinfeld" is about to start in 1 minute and 
turns on the television to the appropriate channel. If the user does not acknowledge the reminder, the 
IEUID will instruct the VCR to record the show. 

It is noted that the reminder can be based on any information, including but not limited to: 

• Content preferences (for example, 'remind me when any movie I have expressed a preference for is 
about to start') 

• Specified reminders (for example, 'Remind me when this Wednesday's episode of "The X-Files" on 
"Global" is about start*, 'Remind me to watch this channel in 1 hours time') 

• Content properties (for example, 'Remind me when any "Seinfeld" episode is about to start 9 , 'Remind 
me when any movie starring Clint Eastwood is about to start 5 ) 

It is noted that the IEUID can be configured in such a way as to automatically remind the user whenever 
any preferred content is about to start. 

It is noted that all parameters of the reminder may be configured by the user including, but not limited to: 

• Reminder offset (for example, '5 minutes before a movie starts', 6 1 minute after "Seinfeld" starts*) 

• Reminder action (for example, 'beep once', 'turn the television on to the correct channel', 'record the 
show if I don't acknowledge the reminder 5 ) 

Content Schedules 

Example: The user wishes to create a calendar for the coming week that lists the shows he is interested in. 
The user uses the Programming System to specify the programs to list on the calendar, and then prints it out 
using a printer attached to the Programming System. 
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It is noted that the Programming System can use any information to assist the user in creating the schedule 
including, but not limited to: 

• Upcoming schedule information (for example, television programming obtained from TVGuide.com) 
» Content information (for example, movie reviews and cast lists from MovieReviews.com) 

• Content preference information obtained previously using the IEUID or Programming System 

• Interactions with the Programming System (for example, responses to computer dialogs that query for 
additional information) 

Interactive content guide 

Example: The user is watching television and wishes to see what television shows are currently playing. 
The user indicates this to the IEUID, and the IEUID displays a list of television shows. The user can scroll 
through the list, and by selecting a show direct the IEUID to instruct the television to change to that 
channel. 

Example: The user is watching television and wishes to see what television shows are playing in one hours 
time. The user indicates this to the IEUID, and the IEUID displays a list of television shows. The user can 
either change to the desired channel immediately, or choose to be reminded when the desired show begins. 

It is noted that the user may filter the list of available content in any way, including but not limited to 

• Time (for example, 6 View all shows currently playing', 4 View all shows playing this week 5 ) 

• Content provider (for example, 'View all shows playing on "Global" 'View all shows playing on FM 
radio stations') 

• Content parameters (for example, 'View all movies', * View all shows of "The Simpsons'", 'View ail 
movies made by "Alfred Hitchcock'") 

• Preferences (for example, 'View all television channels I have expressed a preference for 1 , View all 
radio shows ranked by order of how frequently I listen to each show') 

• Any combination of the above. 

It is noted that the user may interactively alter the filter. For example, the user may be able to use the 

buttons on the IEUID to alter the time for which the television show schedule is displayed. 

It is noted that the content information may be acquired by the IEUID in any way, including but not limited 

to: 

• Configuration via the Programming System (for example, downloading the schedule from 
TVGuide.com) 

• Communication with the Device Network (for example, receiving the current schedule via hrDA from 
a WebT V device) 
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Content Marking 

Example: The user is watching television and an advertisement for the "Abdominizer" comes on. The user 
wants to find out more about the product, or buy the product, and indicates this to the IEUID. The IEUID 
records the relevant information and when the user next synchronizes with the Programming System, he is 
presented with additional information about the advertisement, including a link to the product's web page. 
Example: The user is watching television and an advertisement for the movie "Terminator 2" comes on. 
The user wants to view this movie at a later date and indicates this to the IEUID. The IEUID records the 
relevant information and when the user next synchronizes with the Programming System, the movie 
"Terminator 2" is added to his "Movies I want to see" list. 

Example; The user is listening to radio station preset 1, hears a song that he wishes to purchase the album 
for, and indicates this to the IEUID. The IEUID records the fact that the user was using device 3 on preset 
channel 1 at 1 1:05.37am on January 1, 2000. When the user next synchronizes, the Programming System 
maps the device and channel to the radio station CFNY, and performs a lookup to determine that the artist 
was Marian Carey, and the song was "Heartbreaker" The Programming System then determines all 
albums containing that song, and provides the user with links to web pages where the user can purchase 
those albums. 

It is noted that the IEUID can associate any input from the user with the content mark, including but not 
limited to: 

• Event information (for example, 1 am interested in this product 9 ) 

• Choice information (for example, 'I choose "The X-Files" series ending C) 

• Arbitrary information (for example, "The answer is 3.141 ') 

It is noted that the IEUID and Programming System may use any means to correlate the content mark with 
the stimulus for the content mark, including but not limited to; 

• Chronology (for example, a time stamp) 

• Location (for example, Mississauga, Canada, L5B 3 Y9) 

• Device Network information (for example, currently watching cable television channel 50) 

• External device information (for example, information transmitted from a WebTV device to the 
IEUID) 

It is noted that the IEUID and Programming System may use the content mark and content mark stimulus 
in any way including, but not limited to: 

• Printing a list of movies that the user wishes to view that can be taken to the video rental store. 

• Displaying a list of products in which the user has expressed an interest 

• Displaying a list of additional information based on the content mark stimulus (for example, electronic 
recipes of dishes seen prepared on television). 

• Registering the users *vote* for a particular ending for a television series, 

• Setting reminders to view television shows for which the user has expressed an interest. 
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• Taking the user to the web page for a music artist whose song the user listened to. 

• Recording a list of movies that the user wishes to view on the IEUID and enabling the IEUID to 
communicate with the video store computer to suggest a movie rental that is both available and desired 
by the user. 

It is noted that there may be an indicator present in the content that indicates to the user that the content is 
enhanced for use with the IEUID including, but not limited to: 

• Audible cues (for example, a radio announcer may say "Press a button now on your IEUID to find out 
more information about this album") 

• Visual cues (for example, a television exercise show set on the beach may display text and/or icons in 
the corner of the screen to indicate to the user that by using the IEUID they can obtain more 
information about "Nike Sportswear", "Claudia Schiffer Exercise Videos" and "Barbados Vacations"). 

User content guide 

Example: The user wishes to play 'relaxing music* during a dinner party, and indicates this to the IEUID. 
The IEUID has been previously configured to associate 'relaxing music' with CDs available in group 5 of 
the CD jukebox, and so instructs the AV amplifier to switch to the CD input, instructs the CD jukebox to 
begin playing all CDs in group 5, and sets the volume on low. 

Example: The user wishes to play a particular song from their MP3 collection, and indicates this to the 
IEUID. The IEUID displays a list of available songs on the MF3 player, and the user selects his choice. 
The IEUID translates the choice "Mariah Carey - Heartbreaker" to song number 1354, and instructs the 
MP3 player to begin playing that song. 

Example: The user wishes to play a particular sequence of songs from their CD jukebox, and indicates this 
to the IEUID. The IEUID allows the user to select from the list of available songs, and then programs the 
CD jukebox to play the songs in sequence. 

It is noted that the user may filter the list of available content in any way, including but not limited to: 

• Host device (for example, View all songs on the CD jukebox') 

• Content parameters (for example, 4 View all songs that are waltzes'. 'View all songs by "Mariah 
Carey'") 

• Preferences (for example, 4 View all movies by directors I have expressed a preference for', 'View all 
songs ranked by order of how frequently I listen to each artist') 

• User actions (for example, * View all movies that I have videotaped') 

• Time (for example, 'View all content I have acquired in the last week*) 

• Any combination of the above. 

It is noted that the user content information may be acquired by the IEUID in any way, including but not 
limited to: 
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• Configuration via the Programming System (for example, the user manually enters the MP3 songs they 
own) 

• Communication with the Device Network (for example, querying a CD jukebox for the list of CDs 
contained) 

It is noted that the user content is not required to physically reside within the Device Network (for example, 
the user may select from a list of movies that he has recorded, and be prompted by the IEIJTD to insert tape 
number 3 1 into the VCR, whereupon the IEUID will instruct the VCR to play the second movie on the tape 
by first fast forwarding to a particular position). 

Content databasing 

Example; The user wishes to maintain a database of television shows that have been recorded by the user. 
The user indicates to the IEUID whenever a recordable tape is inserted in the VCR, along with the tape 
identification number. Whenever the IEUID is used to record a show, it keeps track of which show was 
recorded at which position on which tape. The user is then able to scroll through a list on the IEUID of ail 
recorded shows, and can indicate to the IEUID that he wishes to play a particular show. The IEUID 
instructs the user which tape to insert in the VCR, and plays the show by first fast forwarding to the 
appropriate position on the tape. 

It is noted that any type of content (for example, television, radio) on any type of media (for example, video 
tape, audio tape, MiniDisc, computer hard drive) may be databased and accessed in this way. 
It is noted that the user can use the IEUID and Programming System to interact with the database in any 
way (for example, entering information about movies purchased on pre-recorded video tapes, or specifying 
which shows on which tapes can be recorded over in the future). 

It is noted that the databasing information may be stored in the IEUID, the Programming System, or a 
combination thereof 

Device Operation 

Example: The user wishes to watch television and indicates this to the IEUID. The IEUID instructs the 
Device Network in such a way as to allow the user to watch television. While watching television, the 
operation of the IEUID reflects the user's preferences. When the user presses the up and down buttons, the 
IEUID instructs the television to change to channels that match those in the user' s preference list. When 
the user presses the volume up or down button, the IEUID instructs user's AV amplifier to adjust its 
volume (rather than the television volume). When the user presses the navigation down button, the IEUID 
instructs the television to switch to the previously viewed channel 

Example: The user wishes to play a video tape and indicates this to the IEUID. The IEUID instructs the 
Device Network in such a way as to allow the user to play a video tape. While watching the video, the 
operation of the IEUID reflects the user's preferences. When the user presses the volume up or down 
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button, the ffiUID instructs user's AV amplifier to adjust its volume (rather than the television volume). 
When the user presses the navigation left button, the IEUID instructs the VCR to begin rewinding. When 
the user presses the mute button, the IEUID instructs the VCR to pause, and mutes the volume on the AV 
amplifier. 

It is noted that the IEUID may have to maintain information about the state of one or more of the devices ir 
the Device Network (for example, the current cable television channel being viewed is 50, the AV receiver 
is currently set to CD input) 

It is noted that the functionality of the IEUID may be affected in any way by the current state of the IEUID 
including, but not limited to: 

• User input functionality (for example, when watching a video tape, the navigation down button may 
stop the tape, whereas when watching television, the navigation down button may switch to the 
previously viewed channel) 

• User output functionality (for example, when watching television, a list of available cable television 
channels may be shown, while when listening to the radio a list of available radio stations may be 
shown) 

• Internal operation (for example, different infrared codes may be sent to control a particular device 
based on the previous infrared codes sent to the device) 

It is noted that the user output module 6 may be used to assist the user in the operation of the IEUID. For 
example, pressing a particular button may bring up a screen on the LCD that contains images of the 
buttons, along with text describing the function of each button. Pressing the button again may bring up 
another screen with further helpful information. 

Macros 

Example: The user wishes to automate a particular sequence of operations, and indicates this to the IEUID. 
The IEUID prompts the user to enter the sequence of operations, and then prompts the user to assign a 
name for the operation. The user can then choose where within the menuing structure the command(s) that 
activate the sequence of operations should be placed. 

It is noted that macros can be defined using the IEUID, the Programming System, or any combination 
thereof 
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